Manipulating sub-pictures of a compressed video signal

ABSTRACT

The invention relates to manipulation of sub-pictures ( 303 ) in a picture such as a mosaic screen ( 301 ). An apparatus comprises a sub-picture generator ( 105 ) connected to a mosaic screen generator ( 107 ), which generates the mosaic screen  (301 ) from the sub-pictures. The mosaic screen ( 301 ) is by a video compressor ( 111 ) divided into uncompressed picture blocks such that each block only comprise video data from one sub-picture. The picture blocks are compressed using a block compression scheme such as MPEG-2. The apparatus further comprises a video manipulation processor ( 113 ), which manipulates the sub-pictures by manipulating the association between control data in the video signal and the compressed picture blocks. The video data of the compressed picture blocks are not affected whereby de-compression and new compression is avoided.

FIELD OF THE INVENTION

The invention relates to manipulation of sub-pictures in a compressedvideo signal.

BACKGROUND OF THE INVENTION

In order to reduce the bandwidth required to transmit digital videosignals, it is well known to use video compression whereby the data rateof a video signal is substantially reduced.

One of the most widely used video compression techniques is known as theMPEG-2 (Motion Picture Expert Group). MPEG-2 is a block basedcompression scheme, wherein a frame is divided into a plurality ofblocks each comprising eight vertical and eight horizontal pixels. Forcompression of luminance data, each block is individually compressedusing a Discrete Cosine Transform (DCT) followed by quantization, whichreduces a significant number of the transformed data values to zero. Forcompression of chrominance data 1, 2 or even 4 blocks per colourdifference signal are used per macroblock, depending on the coloursub-sampling mode. Frames based only on intra-frame compression areknown as Intra Frames (I-Frames).

In addition to intra-frame compression, MPEG-2 uses inter-framecompression to further reduce the data rate. Inter-frame compressionincludes generation of predicted frames (P-frames) based on previousI-frames or P-frames. In addition, I and P frames are typicallyinterposed by Bidirectional predicted frames (B-frames), whereincompression is achieved by only transmitting the differences between theB-frame and surrounding I- and P-frames frames or surrounding P frames).In addition, MPEG-2 uses motion estimation wherein the image of blocksof one frame found in subsequent frames at different positions arecommunicated simply by use of a motion vector.

As a result of these compression techniques, video signals of standardTV studio broadcast quality level can be transmitted at data ratesaround 2-4 Mbps.

A frequently performed operation by a user of video equipment isnavigation through a video sequence to locate specific occurrence.Current analogue Video Cassette Recorders have fast search modes andslow motion to support the user in finding a specific occurrence. Indigital storage devices, navigation becomes possible based on mosaicscreens, wherein a picture contains a number of sub-pictures that have atemporal distance. In this way, the user can get an overview of aspecific part of a video sequence, and by scrolling through the mosaicscreen, specific sections of the video sequence can be selected. Themosaic screen may comprise sub-pictures with a constant interval or mayrelate to specific events such as a scene change. Specifically,sub-pictures and mosaic screens may be used in a hierarchical fashion tobuild up a Visual Table Of Content (VTOC) which significantlyfacilitates the navigation through video material.

The VTOC could be generated during recording or playback of the videomaterial. At regular intervals or at specific occurrences, the currentvideo image is sampled and stored. The image is then reduced in size andresolution to fit a sub-picture. The mosaic screen generated from aplurality of these sub-images is uncompressed and therefore is verylarge and requires a large bandwidth and storage capacity. Consequently,the mosaic screen is compressed using the MPEG-2 compression algorithm.

When a sub-picture of a mosaic screen is reused in a different mosaicscreen, the compressed mosaic screen is de-compressed, the appropriatesub-picture(s) is/are selected and combined with new sub-pictures.

It is clear, that manipulation of compressed sub-pictures in compressedvideo signals is very complex and requires significant amount ofprocessing power. Thus, manipulation becomes very slow and furtherrequires extra power consumption of the processing resources required.Hence, an improved method of manipulation of sub-pictures in acompressed video signal domain would be advantageous.

SUMMARY OF THE INVENTION

Accordingly, the Invention seeks to improve the manipulation ofsub-pictures of a compressed video signal. Preferably, the inventionalleviates or mitigates one or more of the above disadvantages singly orin combination.

Accordingly there is provided, in a first aspect, a method ofmanipulating sub-pictures of a compressed video signal, comprising thesteps of: generating a video signal comprising a plurality ofsub-pictures; dividing at least a first frame of the video signal into aplurality of uncompressed picture blocks such that each uncompressedpicture block comprises video data related to only one sub-picture;generating a compressed video signal by using a video block basedcompression scheme to generate compressed picture blocks from theuncompressed picture blocks; and manipulating at least a firstsub-picture of the compressed video signal by manipulating theassociation of control data with compressed picture blocks related tothe first sub-picture without modifying compressed video data ofcompressed picture blocks.

Thus, the method provides a very simple method for manipulatingsub-pictures without requiring the compressed video data of thecompressed picture blocks to be modified. Thereby the manipulationoperations can be performed using the existing compressed pictureblocks, and there is no need for the manipulation to include compressionand decompression. This provides for a very low complexity system withhigh flexibility. The computational resource required for manipulationis substantially reduced, and likewise is the delay caused by anymanipulation. Further, as the computational resource is limited, thepower consumption can be reduced as well.

According to one feature of the invention, the step of manipulating atleast the first sub-picture comprises replacing compressed pictureblocks of the first sub-picture with compressed picture blocks of adifferent picture without changing the control data. This provides for avery simple and low complexity method of manipulating the sub-pictures,especially when the compressed picture blocks in the video signal areordered with respect to their position in the full picture.

According to a second feature of the invention, the step of manipulatingthe at least one first sub-picture comprises associating the controldata of a second sub-picture with the compressed picture blocks of thefirst sub-picture. This provides for a very simple and low complexitymethod of manipulating the sub-pictures, especially when the position ofthe compressed picture blocks in the full picture is determined frominformation in the control data.

According to a third feature of the invention, the control datacomprises information related to the position of an associatedsub-picture. This facilitates the manipulation of a position of thesub-picture in the fall picture as this information can be modified toeffect a position change.

According to a fourth feature of the invention, the control datacomprises identification data, and the method further comprises the stepof selecting the compressed data blocks of the first sub-picture byparsing the compressed video signal to detect identification datacorresponding to the first sub-picture. Hence, selection can beperformed by simple parsing, which is a simple, fast and low resourcedemanding operation.

According to a fifth feature of the invention, the compressed videosignal comprises a plurality of slices each slice comprising a sliceheader and a number of consecutive compressed picture blocks. Such anarrangement facilitates the manipulation, as it can be performed bymanipulation of the slices and specifically vertical scrolling can veryeasily be achieved.

According to a sixth feature of the invention, each slice comprises asingle compressed picture block. This provides for a high degree offlexibility of the manipulation, as this can be performed at aresolution as low as the resolution of a single compressed pictureblock.

According to seventh feature of the invention, each slice comprises anumber of compressed picture blocks corresponding to the width of asub-picture. This facilitates handling of sub-pictures, as operations onone slice have effect on the picture across the whole width of thesub-picture.

According to an eighth feature of the invention, each slice comprises anumber of compressed picture blocks corresponding to a width of apicture. This facilitates handling of sub-pictures, as operations on oneslice have effect on the picture across the whole width of the picture.

According to a ninth feature of the invention, the manipulation of theat least first sub-picture comprises manipulating the position of thefirst sub-picture by performing a shifting operation by replacing thecompressed picture blocks of a slice with compressed picture blocks ofanother slice and modifying the control data. This provides for a lowcomplexity method of performing a shifting operation of a picture havingsub-pictures.

According to a tenth feature of the invention, the step of manipulatingat least the first sub-picture comprises manipulating a position of thefirst sub-picture by performing a shifting operation by modifying thecontrol data. This provides for a low complexity method of performing ashifting operation of a picture having sub-pictures.

According to an eleventh feature of the invention, the slice headercomprises a slice number and the manipulation of the at least firstsub-picture comprises manipulating the vertical position of the firstsub-picture by modifying slice numbers for slices comprising compressedpicture blocks of the first sub-picture. Hence, manipulation can beperformed by a simple operation of changing slice numbers. Specifically,a vertical scrolling operation can be achieved by a simple manipulationof slice numbers.

According to a twelfth feature of the invention, the step ofmanipulating the at least first sub-picture farther comprises replacingthe compressed picture blocks with pre-defined compressed picture blockswithout modifying the control data whereby pre-determined sub-picturescan be inserted in the compressed video signal. This provides for asimple method for predetermined pictures such as markers or icons to beinserted in the picture.

Preferably, the block compression scheme is an MPEG-2 scheme and thecompressed picture blocks are macro blocks.

According to a second aspect of the invention, there is provided anapparatus for manipulating sub-pictures of a compressed video signal,comprising: means for receiving a compressed video signal comprising aplurality of sub-pictures; wherein at least a first frame of thecompressed video signal is divided into compressed picture blocks byvideo block compression of uncompressed picture blocks, the uncompressedpicture blocks comprising video data related to only one sub-picture;and means for manipulating at least a first sub-picture of thecompressed video signal by manipulating the association of control datawith compressed picture blocks related to the first sub-picture withoutmodifying compressed video data of compressed picture blocks.

These and other aspects of the invention will be apparent from andelucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the invention will be described, by way of exampleonly, with reference to the drawings, in which

FIG. 1 is an illustration of an apparatus for manipulating sub-picturesof a compressed video signal in accordance with an embodiment of theinvention;

FIG. 2 illustrates a flow-chart of the operation of the apparatus ofFIG.1 in accordance with an embodiment of the invention;

FIG. 3 illustrates an example of a mosaic screen for the preferredembodiment of the invention;

FIG. 4 illustrates a mosaic screen containing sub-pictures constructedwith mini slices wherein the width of a slice is equal to the width of asub-picture;

FIG. 5 illustrates a mosaic screen containing sub-pictures constructedwith slices wherein the width of a slice is equal to the width of thewhole picture; and

FIG. 6 illustrates a mosaic screen containing sub-pictures constructedwith mini slices wherein the width of a slice is equal to the width of acompressed picture block.

FIG. 7 shows a further advantageous coding according to an embodiment ofthe invention.

DESCRIPTION OF PREFERRED EMBODIMENTS

An embodiment of the invention will in the following be described withspecific reference to MPEG-2 compression, but it will be apparent thatthe invention is not limited to this application and may be equallyapplicable to many other video block based compression schemes.

FIG. 1 is an illustration of an apparatus 100 for manipulatingsub-pictures of a compressed video signal in accordance with anembodiment of the invention. FIG. 2 illustrates a flow-chart of theoperation 200 of the apparatus 1 00 of FIG. 1 in accordance with anembodiment of the invention.

FIG. 1 shows a video source 101 connected to a picture sampler 103comprised in the apparatus 100. The video source may be an externalcamera, video storage unit or a receiver for a broadcast video signal,or any other suitable video source. As such, the video source 101 may beexternal or internal to the apparatus 100. The video signal from thevideo source 101 is digital, and is in the preferred embodiment adigital video signal compressed using an MPEG-2 compression scheme. Thepicture sampler 103 is connected to a sub-picture generator 105, whichis further connected to a mosaic screen generator 107. The apparatus 100further comprises a mosaic screen controller 109, which is connected tomost of the other functional blocks of the apparatus 100. The mosaicscreen controller 109 is operable to control the various functionalblocks of the apparatus 100 to perform the required operations for thegeneration and manipulation of the mosaic screens. The mosaic screencontroller 109 receives user input and controls the operation of theapparatus accordingly.

In step 201 of the flow chart of FIG. 2, a sub-picture is generated. Thesub-picture is generated from the video signal from the video source101. When the mosaic screen controller 109 determines that a sub-pictureshould be generated, it transmits a control signal to the picturesampler 103, which in return samples the picture currently beingreceived from the video source. In the preferred embodiment, wherein thevideo signal is an MPEG-2 compressed video signal, the samplingcomprises storing all information relevant to decode the current imagefrom the video signal. As such, this information may simply comprise allinformation from an I-frame, but in other situations it may include theanchor pictures in order to properly decode a selected predictivepicture. The sampled data values are passed to the sub-picturegenerator. In the preferred embodiment, the picture sampler 103 storesthe data in a memory that can also be accessed by the sub-picturegenerator 105. When the picture sampler has extracted and stored allnecessary information, it transmits a control signal to the mosaicscreen controller 109, which in return starts the sub-picture generator105.

The sub-picture generator 105 generates a sub-picture by decompressingthe frame using the information captured by the picture sampler 103, andin the preferred embodiment, by using MPEG-2 compression techniques asare well known in the art. Once the decompressed image is generated, itis resized and the resolution is changed accordingly. Preferably, thisis achieved by resampling and decimating the original image either inthe frequency domain or in the time domain. In the preferred embodiment,wherein a mosaic screen is made up of 16 sub-pictures with demarcationborder between sub-pictures, the resampling is such that both thehorizontal and vertical pixel resolution is reduced by a factor of four.Specifically a 384 by 320 pixel image is resampled as a 96 by 80 pixelpicture.

In step 203, the mosaic screen controller 109 determines if moresub-pictures must be generated, and if so the method returns to step201. In the preferred embodiment, sub-pictures are generated for theentire received video sequence before any mosaic screens are generated.In other embodiments, the generation of mosaic screens begins as soon asa number of sub-pictures corresponding to the number of pictures in thescreen is generated. In further embodiments, the generation of a mosaicscreen is begun as soon as one sub-picture has been received.

Specifically, the generation of sub-pictures may be generated duringrecording, playback or during idle times of the apparatus. Thus, theapparatus may have functionality for generating the sub-pictures duringrecording or playback, but as this is computationally demanding, theapparatus may have functionality for accessing the recorded video-signaland generating the sub-pictures when the apparatus is otherwise idle.This reduces the requirements for the computational resource, as thesame resource can be used for compression during recording,de-compression during playback and sub-picture gemeration during idletimes.

In the preferred embodiment, the mosaic screen controller 109 generatesa sub-picture at regular intervals such that the temporal distancebetween sub-pictures is constant. It is within the contemplation of theinvention, that sub-pictures may be generated at discrete timesaccording to any suitable algorithm or scheme. Specifically, the mosaicscreen controller may instigate the generation of a new sub-picture whenthe video signal comprises a scene change.

When sufficient sub-pictures for the generation of a mosaic screen havebeen generated, the process continues in step 205 by generating a videosignal comprising sub-pictures. In the preferred embodiment, the mosaicscreen generator 107 generates a video signal comprising a mosaic screenof sub-pictures.

FIG. 3 illustrates an example of a mosaic screen 301. The mosaic screen301 has a pixel height of Hm pixels and a width of Wm pixels. In apreferred embodiment Hm is 320 pixels and Wm is 384 pixels. In apreferred embodiment, the mosaic screen 301 comprises sixteensub-pictures 303. Any number of sub-pictures that is an integerdivisional by the macroblock size can be used to generate amosaic-screen. Each sub-picture has a pixel height of Hsp pixels and awidth of Wsp pixels. In the preferred embodiment, Hsp is 80 pixels andWsp is 96 pixels. Thus, the mosaic screen is simply made up bypositioning the pixels of a sub-picture adjacent to the pixels of theadjacent sub-picture in the mosaic screen. In the preferred embodiment,the mosaic screen and the sub-pictures are stored as data values inmemory, and the generation of the mosaic screen is achieved by movingthe data values for the respective sub-pictures into the appropriatememory locations of the mosaic screen.

When the video signal comprising the mosaic screen has been generated instep 205, the method continues in step 207 and 209 by compressing thevideo signal. In the preferred embodiment, the compression scheme usedis MPEG-2 and thus the same compression scheme is used for compressingthe mosaic screen as was used for the original video signal. Thecompression is performed in the video compressor 111 that receives thepicture comprising the mosaic screen from the mosaic screen generator107. In the preferred embodiment, the transfer of the mosaic screen isby a pointer to the location of the mosaic screen in the memory.

In step 207, at least a first frame of the video signal is divided intoa plurality of uncompressed picture blocks such that each picture blockcomprises video data related to only one sub-picture. Thus, in thepreferred embodiment, the picture is divided into a number of macroblocks of 16 by 16 pixels as appropriate for the MPEG-2 compressionscheme. The division is further such that each macro block is entirelywithin one sub-picture and thus the border of the sub-pictures isaligned with the borders of the macro-blocks. Hence, each sub-picture of96 by 80 pixels is divided into 30 macro blocks arranged with 6horizontally by 5 vertically. As a consequence, any direct operation ona sub-picture will only affect the macro-blocks of that sub-picture andwill not affect any other sub-picture. In the preferred embodiment, theborders of the sub-pictures coincide with the edges of the macro-blocks,but in other embodiments, the sub-pictures are comprised within theboundaries of the macro-blocks but without the edges aligning. In thiscase, the remaining pixels can comprise a border separating theindividual sub-pictures in the mosaic screen.

In step 209, a compressed video signal is generated by using a videoblock compression scheme to generate compressed picture blocks from theuncompressed picture blocks. Thus, in the preferred embodiment, MPEG-2is used as the block compression scheme, and the mosaic screen iscompressed using the MPEG-2 compression algorithm based on the divisioninto macro blocks as performed in step 207. Block compression usingMPEG-2 is well known in the art and will for brevity not be describedfurther here.

The compressed video signal is fed to a video manipulation processor 113which is operable to perform various operations on the compressed mosaicscreen including for example scrolling, shifting, sub-picture selection,sub-picture replacement etc. The video manipulation processor 113 isconnected to the mosaic screen controller, which through a userinterface is operable to control the video manipulation processor 113from user input.

Following the compression step 209, the method in accordance with thepreferred embodiment continues in step 211, wherein at least a firstsub-picture of the compressed video signal is manipulated bymanipulating the association of control data with compressed pictureblocks related to the first sub-picture without modifying compressedvideo data of compressed picture blocks.

Following the compression, each of the uncompressed picture blocks hasbeen compressed to a compressed picture block, and thus each compressedpicture block retains the property of only comprising video data relatedto a single sub-picture. Hence, sub-pictures can be manipulated in themosaic screen by manipulation of the control data of the compressedpicture blocks (macro blocks) without any amendment to the compressedvideo data of the macro blocks.

The video signal comprises control data associated with the compressedpicture blocks. This control data comprises various information data,typically including information related to the location of thecompressed picture blocks within the mosaic screen. Hence, a sub-picturecan be moved within the mosaic screen simply by changing the locationinformation of the control data associated with the compressed pictureblocks of that sub-picture. In the preferred embodiment of an MPEG-2compression scheme, the control data specifically comprises slicenumbers contained in slice headers and indicating a vertical position,as well as macro block control data contained in the macro block andindicating a horizontal position. Thus, the control data may be acontrol data part of the compressed picture blocks (i.e. a data blockhaving control data rather than picture data). Alternatively, thecompressed picture blocks can be considered to correspond to the picturedata part of the MPEG-2 macro blocks only. Thus, the control data ofMPEG-2 macro blocks may be considered as not belonging to the compressedpicture blocks.

In accordance with one embodiment, the manipulation of a firstsub-picture comprises replacing compressed picture blocks of the firstsub-picture with picture blocks of a different picture without changingthe control data. Consequently, the first sub-picture is replaced in themosaic screen by the second sub-picture simply by replacing whole macroblocks or the picture data part of macro blocks without anyde-compression or compression being required. In fact, simple parsingcan be used to identify macro blocks of the first and the second blocks.The control data is not itself changed but the data content of blocksassociated with the control data is changed from being data of the firstsub-picture to being data of the second sub-picture. This isparticularly useful for embodiments, wherein the video signal protocolis such that compressed picture blocks are ordered in the video signalaccording to their position within the picture, and where control datais followed by the compressed picture blocks to which it relates.

In a different embodiment of the invention, the step of manipulating thesub-picture comprises associating the control data of a secondsub-picture with the compressed picture blocks of the first sub-picture.In this embodiment, a horizontal relocation of a sub-picture within amosaic screen can be achieved by changing data in the control data suchthat the relation is changed from being to the macro-blocks of the firstpicture to being to macro-blocks of the second picture. This isparticularly advantageous for embodiments wherein compressed pictureblocks are not ordered in the video signal according to their positionwithin the picture but the location is determined by locationinformation in the associated control data.

In the preferred embodiment where MPEG-2 compression is used, thecompressed video signal comprises a plurality of slices each slicecomprising a slice header and a number of consecutive compressed pictureblocks. This enables the manipulation of the sub-pictures to beperformed on the slices and specifically by the control data comprisedin the slice headers.

Various strategies for sub-picture encoding with slices can be usedincluding each slice comprising a single compressed picture block, anumber of compressed picture blocks corresponding to a width of asub-picture or a number of compressed picture blocks corresponding to awidth of the whole picture.

FIGS. 4 to 6 shows mosaic screens containing sub-pictures constructedwith slices of different sizes. The figures will in the following bedescribed with specific reference to the MPEG-2 compression format,wherein the position data of compressed picture blocks is comprised inthe slice header (vertical position) and the control data of the firstmacro block directly following the slice header (horizontal position).The compressed picture blocks will in the following description refer tothe picture data content of macro blocks, and control data will refer tothe control data of the slice header and/or the control data comprisedin the macro blocks and specifically in the first macro block of eachslice.

FIG. 4 illustrates a mosaic screen containing sub-pictures constructedwith mini slices, wherein the width of a slice is equal to the width ofa sub-picture, i.e. 96 pixels in the preferred embodiment. The height ofthe slice is 16 pixels and thus a sub-picture is made up of 5 slices. Inthis embodiment, the sub-pictures can be filtered from an existingmosaic screen, retrieved from a sub-picture data base or relocated to adifferent position simply by operating on the mini slices and thecontrol data of the macro blocks without changing any compressed picturedata. For example, by changing the association of control data of sliceheaders and the first macro block with compressed picture blocks(comprised by the picture data of the macro blocks), the position of thecompressed picture blocks may be changed within the mosaic screen. Thechange in association can be by changing the slice header data, verticalshift and/or control data of the macro block's horizontal shiftinformation. Alternatively or additionally, the change in associationcan be managed by manipulating the macro blocks within the signal suchthat the macro blocks following a slice header are changed.

In MPEG-2, a slice header comprising the vertical information, isfollowed by a macro block comprising horizontal information. Thefollowing macro blocks do not comprise absolute horizontal informationand therefore the second and consequent macro blocks may be moved in thepicture without the control data of slice headers and first macro-blocksbeing changed. Hence, the slice headers and first macro blocks can beseen as a template into which the various compressed picture blocks canbe put. In FIGS. 4-6, the arrows thus illustrate the position of theslice headers. As such, FIG. 4 may be seen as illustrating the templatefor the mosaic screen, which can then be filled with appropriatecompressed picture blocks.

In different embodiments, the sub-pictures may be manipulated by forexample only performing operations on slice headers, on macro blocks oron both slice headers and control data of the macro-blocks.Alternatively and additionally, the manipulation may be by manipulationof macro-blocks within the template formed by the slice headers and thehorizontal information control data of the first macro-block of eachslice.

FIG. 4 illustrates an example of a horizontal shift in a mosaic screen.In this embodiment, the position of the first sub-picture is manipulatedby replacing the compressed picture blocks of a slice with compressedpicture blocks of the consecutive slice without modifying the sliceheader or the control data of the macro blocks directly following theslice header. Thus, the picture data of the macro blocks of one slice isreplaced by the picture data of the macro blocks of the consecutiveslice. Hence, the compressed picture blocks in slice 401 are moved toslice 403, while the control data of the slice is maintained, i.e. theslice header and the control data of the first macro blocks of bothslice 401 and 403 are unchanged. Likewise, the compressed picture blocksof slice 403 are moved to slice 405, the compressed picture blocks ofslice 405 to slice 407 and so on. The compressed picture blocks of theslice 407 are moved to slice 409. New compressed picture blocks areadded to slice 401. In one embodiment, the new compressed picture blockssimply correspond to a black image. In other embodiments, the newcompressed picture blocks inserted in slice 401 may represent a newsub-picture. In yet other embodiments, the picture data of the slice 407is not moved to slice 409, but is moved to slice 401 resulting in ahorizontal rotation.

In the embodiment of the previous paragraph, the sub-picture on theright-hand side of the screen is moved to the left-hand side of themosaic screen but lowered one slice level (or at the same slice level ifcompressed picture blocks of slice 407 are moved to slice 401 etc.). Ina typically preferred embodiment, the sub-picture on the right hand sideof the mosaic screen is moved to the position of the next lowersub-picture on the left-hand side of the mosaic screen. In thisembodiment, slices from the sub-picture(s) on the right-hand side of themosaic screen are moved to slices of the left-hand sub-pictures, whichare as many slice levels lower as there are vertical slice levels in asub-picture. Thus, in the example described wherein each sub-picturecomprises five slices, the compressed picture blocks of slice 407 aremoved to the slice five levels below slice 401.

FIG. 5 illustrates the embodiment of a slice having a width identical tothe width of the picture. The scrolling operation is here achieved bythe compressed picture blocks of slice 501 being moved to the slice 503of the next line and so forth. In this example, the horizontal positionof the macro blocks is unchanged and only the vertical position ischanged. Hence, in this case the control data of the macro blocks areidentical for the horizontal position data, and therefore the controldata of the macro blocks may be moved together with the picture data ofthe macro blocks. Thus, the compressed picture blocks may in thisexample be identical to the macro blocks of the MPEG-2 compressionscheme. In this case, the entire mosaic screen is thus scrolled down (orup) with very low complexity. This embodiment requires modification ofslice headers only and does not require modification of control data ofthe macro blocks. However, manipulation of the mosaic screen is limitedto the vertical direction as the slice headers of the preferredembodiment only comprise vertical position information.

FIG. 6 illustrates an embodiment wherein each slice comprises only asingle compressed picture block. A shifting operation is achieved bymoving the compressed picture blocks (i.e. the picture data of the macroblocks) from slice 601 to 603, from 603 to 605 and so on, whilemaintaining the control data of the slice header and macro blocks. Aslice size of only a single compressed picture block provides increasedflexibility as the manipulation can be done on a very small pictureblock but increases the complexity as more slice headers and controldata must be manipulated. Similarly to the example of FIG. 4, thecompressed picture blocks of the right-hand slices are preferably movedto the left-hand side slices five levels below.

A number of other operations can be performed on the sub-pictures basedon manipulation of slice headers. In one embodiment, the slice headercomprises a slice number and the manipulation of the at least firstsub-picture comprises manipulating the position of the first sub-pictureby modifying slice numbers for slices comprising compressed pictureblocks of the first sub-picture. This specifically provides for thefunctionality of moving sub-pictures in a vertical direction.

In accordance with an embodiment of the invention, the control datacomprises identification data, and the method further comprises the stepof selecting the compressed data blocks of the first sub-picture byparsing the compressed video signal to detect identification datacorresponding to the first sub-picture. Thus, the filtering of asub-picture from an existing compressed mosaic screen can be achieved bya low complexity material. In this case, the video signal is simplyparsed for identity information corresponding to the sub-picture to beselected. Specifically, the slice headers and/or control data of themacro blocks are parsed and the compressed picture blocks of the sliceshaving the corresponding identity information are extracted.

Similarly, a new mosaic screen can be created by low complexity methods.For example in FIG. 3, replacing sub-picture 2 with sub-picture 6 can beachieved by simply replacing the slice numbers in the slice headers ofthe corresponding slices. This number is a byte aligned eight bit numberthat can easily be identified by parsing the video signal.

Further, the complex and resource demanding creation of sub-picturesneeds only to be performed once and the generated sub-pictures can bere-used for various purposes by filtering as described above.

In accordance with one embodiment, the method includes replacing thecompressed picture blocks with pre-defined compressed picture blockswithout modifying the control information, whereby pre-determinedpictures can be inserted in the compressed video signal. Hence, thisallows easy insertion of synthetic macro blocks such as markers, icons,cursors etc. For example, a picture corresponding to a marker or text ofdifferent font can be created and compressed using MPEG-2 compression.The resulting compressed picture blocks can be stored in memory andlater retrieved and inserted into a mosaic screen by directly replacingthe appropriate compressed picture blocks of the video signal with thosestored. Thereby pre-determined sub-pictures can easily be inserted inthe mosaic screen.

FIG. 7 shows an advantageous embodiment of the invention, wherein afurther mosaic screen 701 is predictively coded with reference to mosaicscreen 301, which is used as an anchor picture. The mosaic screen 301may be encoded as an I picture or a P picture. For many applications, agiven sub-picture in the further mosaic picture 701 is efficiently codedby setting all block motion vectors of the given sub-picture to a valuewhich is equal to the relative sub-picture displacement between mosaicscreen 701 and the reference screen 301. The relative sub-picturedisplacement can be determined under user control or in the mosaicgeneration process at transmitter side. In both cases, motion estimationand block matching is not required, because the sub-picture displacementis set outside the coder. Desired movement of one or more sub-picturescan be obtained from the user e.g. via a mouse or joystick. Thesedesired movements can directly or indirectly be mapped to motion vectorsfor the corresponding blocks. As an example, FIG. 7 shows a scrollingoperation. In screen 701 all sub-pictures 5 . . . 16 are moved upwardsover a distance corresponding with the height of the sub-pictures. Newsub-pictures 17 . . . 20 are added at the bottom of the screen 701. Allmotion vectors in the blocks of the sub-pictures are set equal to thedisplacement of the sub-pictures which is in this case the same for allsub-pictures. For example, all motion vectors of the blocks insub-picture 5 are equal to the displacement mv5 of sub-picture 5 inmosaic screen 701 with reference to sub-picture 5 in mosaic screen 301.Depending on the operation performed, several sub-pictures can beencoded in such a way. Sub-pictures 17 . . . 20 for which nocorresponding sub-pictures are available in the mosaic screen 301 can beintra coded, e.g. in the form of intra-coded macroblocks. If there issome correlation between blocks in the sub-pictures 17 . . . 20 andblocks in mosaic screen 301, the blocks in the sub-pictures 17 . . . 20can also be predictively coded with reference to blocks in the mosaicscreen 301.

The above description has for brevity and clarity focussed on anembodiment for mosaic screens but it will be apparent that the inventionis equally applicable to other screens, images or pictures havingsub-pictures.

The invention can be implemented in any suitable form includinghardware, software, firmware or any combination of these. However,preferably, the invention is implemented as software program running onone or more data processors. The functionality, elements and componentsmay be implemented in a single unit, in a plurality of units or as partof other functional units.

Although the present invention has been described in connection with thepreferred embodiment, it is not intended to be limited to the specificform set forth herein. Rather, the scope of the present invention islimited only by the accompanying claims.

1. A method of manipulating sub-pictures of a compressed video signal,comprising the steps of: generating a video signal comprising aplurality of sub-pictures; dividing at least a first frame of the videosignal into a plurality of uncompressed picture blocks such that eachuncompressed picture block comprises video data related to only onesub-picture; generating a compressed video signal by using a video blockbased compression scheme to generate compressed picture blocks from theuncompressed picture blocks; and manipulating at least a firstsub-picture of the compressed video signal by manipulating theassociation of control data with compressed picture blocks related tothe first sub-picture without modifying compressed video data ofcompressed picture blocks.
 2. A method as claimed in claim 1 wherein thestep of manipulating at least the first sub-picture comprises replacingcompressed picture blocks of the first sub-picture with compressedpicture blocks of a different picture without changing the control data.3. A method as claimed in claim 1 wherein the step of manipulating atleast the first sub-picture comprises associating the control data of asecond sub-picture with the compressed picture blocks of the firstsub-picture.
 4. A method as claimed in claim 1 wherein the control datacomprises information related to the position of an associatedsub-picture.
 5. A method as claimed in claim 1 wherein the control datacomprises identification data and the method further comprises the stepof selecting the compressed data blocks of the first sub-picture byparsing the compressed video signal to detect identification datacorresponding to the first sub-picture.
 6. A method as claimed in claim1 wherein the compressed video signal comprises a plurality of sliceseach slice comprising a slice header and a number of consecutivecompressed picture blocks.
 7. A method as claimed in claim 6 whereineach slice comprises a single compressed picture block.
 8. A method asclaimed in claim 6 wherein each slice comprises a number of compressedpicture blocks corresponding to a width of a sub-picture.
 9. A method asclaimed in claim 6 wherein each slice comprises a number of compressedpicture blocks corresponding to a width of a picture.
 10. A method asclaimed in claim 6 wherein the step of manipulating at least the firstsub-picture comprises manipulating a position of the first sub-pictureby performing a shifting operation by replacing the compressed pictureblocks of a slice with compressed picture blocks of another slice and/ormodifying the control data.
 11. A method as claimed in claim 6 whereinthe step of manipulating at least the first sub-picture comprisesmanipulating a position of the first sub-picture by performing ashifting operation by modifying the control data.
 12. A method asclaimed in claim 6 wherein the slice header comprises a slice number andthe manipulation of the at least first sub-picture comprisesmanipulating the vertical position of the first sub-picture by modifyingslice numbers for slices comprising compressed picture blocks of thefirst sub-picture.
 13. A method as claimed in claim 1 wherein the stepof manipulating the at least first sub-picture further comprisesreplacing compressed picture blocks with pre-defined compressed pictureblocks without modifying the control data whereby pre-determinedsub-pictures can be inserted in the compressed video signal.
 14. Amethod as claimed in claim 1, wherein a given sub-picture ispredictively coded and wherein all block motion vectors associated withthe plurality of picture blocks within the given sub-picture are set toa same value which depends on a displacement of the given sub-picturebetween a picture and a reference picture.
 15. A method as claimed inclaim 14, wherein the sub-picture displacement is user-controllable. 16.A computer program enabling the carrying out of a method according toclaim
 1. 17. An apparatus for manipulating sub-pictures of a compressedvideo signal, comprising: means for receiving a compressed video signalcomprising a plurality of sub-pictures; wherein at least a first frameof the compressed video signal is divided into compressed picture blocksby video block compression of uncompressed picture blocks, theuncompressed picture blocks comprising video data related to only onesub-picture; and means for manipulating at least a first sub-picture ofthe compressed video signal by manipulating the association of controldata with compressed picture blocks related to the first sub-picturewithout modifying compressed video data of compressed picture blocks.